Distributed Evolution Strategies for Black-Box Stochastic Optimization

This work concerns the evolutionary approaches to distributed stochastic black-box optimization, in which each worker can individually solve an approximation of problem with nature-inspired algorithms. We propose a evolution strategy (DES) algorithm grounded on proper modification strategies, family classic algorithms, as well careful combination existing frameworks. On smooth and nonconvex landscapes, DES has convergence rate competitive zeroth-order methods, exploit sparsity, if applicable, match first-order methods. The method uses Gaussian probability model guide search avoids numerical issue resulted from finite-difference techniques is also fully adaptive landscape, its guaranteed any parameter setting. further two alternative sampling schemes significantly improve efficiency while leading similar performance. Simulation studies several machine learning problems suggest that proposed methods show much promise reducing time improving robustness settings.